{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Python inference tutorial\n",
    "\n",
    "This tutorial will describe how to use the Inference Process.\n",
    "\n",
    "\n",
    "**Requirements:**\n",
    "\n",
    "* Run the notebook inside the Python virtual environment: ```source hailo_virtualenv/bin/activate```\n",
    "\n",
    "It is recommended to use the command ``hailo tutorial`` (when inside the virtualenv) to open a Jupyter server that contains the tutorials."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standalone hardware deployment\n",
    "\n",
    "The standalone flow allows direct access to the HW, developing applications directly on top of Hailo\n",
    "core HW, using HailoRT. This way the Hailo hardware can be used without Tensorflow, and\n",
    "even without the Hailo SDK (after the HEF is built).\n",
    "\n",
    "An HEF is Hailo's binary format for neural networks. The HEF files contain:\n",
    "\n",
    "* Target HW configuration\n",
    "* Weights\n",
    "* Metadata for HailoRT (e.g. input/output scaling)\n",
    "\n",
    "First create the desired target object.\n",
    "In this example the Hailo-8 PCIe interface is used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from hailo_platform import VDevice, HailoSchedulingAlgorithm\n",
    "\n",
    "timeout_ms = 1000\n",
    "\n",
    "params = VDevice.create_params()\n",
    "params.scheduling_algorithm = HailoSchedulingAlgorithm.ROUND_ROBIN\n",
    "\n",
    "# The vdevice is used as a context manager (\"with\" statement) to ensure it's released on time.\n",
    "with VDevice(params) as vdevice:\n",
    "\n",
    "    # Create an infer model from an HEF:\n",
    "    infer_model = vdevice.create_infer_model('../hefs/resnet_v1_18.hef')\n",
    "\n",
    "    # Configure the infer model and create bindings for it\n",
    "    with infer_model.configure() as configured_infer_model:\n",
    "        bindings = configured_infer_model.create_bindings()\n",
    "\n",
    "        # Set input and output buffers\n",
    "        buffer = np.empty(infer_model.input().shape).astype(np.uint8)\n",
    "        bindings.input().set_buffer(buffer)\n",
    "\n",
    "        buffer = np.empty(infer_model.output().shape).astype(np.uint8)\n",
    "        bindings.output().set_buffer(buffer)\n",
    "\n",
    "        # Run synchronous inference and access the output buffers\n",
    "        configured_infer_model.run([bindings], timeout_ms)\n",
    "        buffer = bindings.output().get_buffer()\n",
    "\n",
    "        # Run asynchronous inference\n",
    "        job = configured_infer_model.run_async([bindings])\n",
    "        job.wait(timeout_ms)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
